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5 Related Applications 

This application incorporates by reference in its entirety and is a divisional of 
U.S. Patent Application No. 09/966,375, titled "SYSTEM AND METHOD FOR 
PROVIDING RANDOM ACCESS TO A MULTIMEDIA OBJECT OVER A 
NETWORK," filed September 27, 2001, which in turn is a divisional of and 
10 incorporates by reference in its entirety U.S. Patent Application No. 09/167,521, filed 
October 6, 1998, now U.S. Patent No. 6,314,466, issued November 6, 2001. 

Background of the Invention 

Field of the Invention 

15 The invention relates to accessing segments of a media object over a network. 

More particularly, the invention provides a system and method for determining a 
preliminary transmission period for a segment of a media object. 
Description of the Related Art 

Streaming media is a method of making audio, video, and other multimedia data 

20 available in real-time, over a network. Figure 1 illustrates a typical streaming media 
system. If the source of the streaming media is a live performance, the performance is 
often recorded as a data signal by an input device, such as, for example, a microphone 
102 or a camera 104. The input device then sends the data signal to an encoding system 
106 which converts the data signal into a digital form and compresses the digital signal 

25 into a streamable data object 107. The streamable data object 107 may be sent to a 

content creation station 108 for editing or may altematively be sent directly to a 
streaming media server 110. The content creation station 108 includes content creation 
software, such as video editing software, that allows the user to modify the streamable 
data object 107. The content creation station 108 can also be used to independently 

30 create a streamable data object 107. After being modified or created by the content 
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creation station 108, the streamable data object 107 may then be transmitted to the 
streaming media server 110. 

Once the streamable data object 107 is located on the streaming media server 
110, the streamable data object 107 is made available for further distribution over a 
5 network 100, such as the Internet, to one or more client computers 112. The client 
computer 112 generally requests access to the streamable data object 107, begins 
receiving portions of it, and begins rendering the streamable data object 107 into a 
multimedia presentation for an end user. 

One problem, however, related to transmitting streamable data objects over a 

10 network is that the client computer 112 must determine when it has sufficient data to 
begin the presentation or playing of the streamable data object 107. One solution has 
been to require the client computer 112 to store the whole presentation in memory 
before beginning the display of the presentation. However, this approach consumes 
significant system resources and creates unacceptable delay between the user's initial 

15 request for the media and the start of the presentation. Also, the client computer 1 12 
may not have the system capacity to store the whole streamable data object 107. 

Under another approach implemented by some existing systems, the client 
computer 112 starts playing the streamable data object 107 as soon as it is received. Of 
the problems with this approach, perhaps the most unacceptable is the periodic freezing 

20 or delay during playback caused by a lack of sufficient data to render the streamable 

data object 107. Bandwidth intensive segments of the streamable data object 107 are 
often responsible for such. To ensure a seamless presentation, the client computer 1 12 
may require more data from the streaming media server 110 than has been transmitted 
over the network 100. But the amount of data a client computer 1 12 can receive in any 

25 period of time is restricted by the type of network communication it uses. 

For example, assuming that the network 100 is the Internet, it is common for a 
client computer 112 to use a modem to connect to the Intemet. However, modems have 
limited bandwidth capabilities compared to other transmission mediums, i.e., television, 
radio, Ethernet. The bandwidth of a communication device is defined by the amount of 

30 data that can be communicated over a specified time. A 28.8 Kbps (kilobits per 
second) modem can transmit or receive around 29,000 bits per second. 
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Depending on the content of a streamable data object 107, the rate at which data 
is consumed during presentation of the streamable data object 107 may be much higher 
than the transmission rate for a particular time interval. In such cases, the client 
computer 112 of a conventional streaming media system simply waits in the middle of 
5 the presentation of the streamable data object 107 for the transmission of additional 
data, creating an unpleasant interruption for the user. 

Figure 2 is a high level block diagram representing the data transmission process 
of the prior art. Starting at a state 200, the client computer 1 12 (Figure 1) requests the 
streamable media server 1 10 (Figure 1) to start transmitting data of the streamable data 

10 object 107. Next, at a state 202, the streaming media server 1 10 starts sending data of 
the streamable data object 107 (Figure 1) to the client computer 1 12. Moving to a state 
204, the client computer 1 12 starts displaying the presentation of the transmitted data. 
Continuing to a state 206, the client computer 1 12 stops the presentation in order to wait 
for further data. Lastly, at a state 208, the client computer 112 finishes displaying the 

15 presentation. Although only one interruption is indicated in the flowchart of Figure 2, 
the client computer 112 may suffer from multiple interruptions due to having 
insufficient data at various times in the rendering process. 

For fijrther example, referring to Figures 3 and 4, a consumption graph for an 
exemplary rendering of a streamable data object 107 is described. Figure 4 shows the 

20 target bit rate of the client computer 1 12 in addition to the consumption graph of Figure 
3. The consumption graph illustrates that the number of bits that are consumed during 
presentation of a streamable data object 107 varies over time. During the first one 
second interval of the presentation, the client computer 112 (Figure 1) consumes 30K 
bits of data. However, during the next second, the client computer 1 12 consumes only 

25 lOK bits of information. Table 1, based on the consumption graph in Figures 3 and 4, 

further illustrates a situation wherein the total number of bits that are needed by the 
client computer 112 to maintain an uninterrupted playback is larger than the total 
number of bits it has received at multiple points in time. 
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Table 1 
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20K 
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80K 
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20K 


20K 


lOOK 


lOOK 


6 


lOK 


lOK 


llOK 


llOK 



As can be seen from Table 1 and Figure 2, after one second, in order to 
seamlessly present the streamable data object 107, the client computer 112 needs 30K 
5 bits; however, the client computer 1 12 has only received 20K bits of the streamable data 
object 107. Therefore, the client computer 112 has to wait for further data before 
starting the presentation. Further, after the presentation is started, the client computer 
1 12 has to halt again because the presentation requires 70K bits of data at three seconds, 
but the client computer has only received 60K bits. This halt manifests itself, in 

1 0 existing systems, by freezing the presentation. 

To attempt to overcome this difficulty, some systems transmit a portion of a 
streamable data object before playback starts. However, all multimedia presentations 
are not always played fi-om the beginning, and transmitting an initial portion of a 
streamable data object does not help to address such situations. 

15 The client computer 112 typically has a rendering program (not shown) that 

allows the user to "seek" to the middle or any number of secondary positions in the 
presentation before or during playback. After starting to play at the secondary position, 
the client computer 112 may require more data for the presentation than has been 
transmitted. Thus, the client computer 112 must pause and wait for the transmission of 

20 additional data. One solution to this problem has been to estimate the initial 
transmission buffer for the secondary position as being equal to or slightly greater than 
the initial transmission buffer for the whole streamable data object. However, this 
approach is merely a rough estimate and cannot guarantee that the client computer 1 12 
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will render the presentation without further interruptions and additional waits for further 
data. Thus, this approach does little to ensure a smooth, uninterrupted playback of a 
media object. 

Variations in rendering requirements (data consumption over time) are more 
5 apparent in low bandwidth media systems, such as MacroMedia Shockwave Flash 2.0. 
MacroMedia Shockwave Flash 2.0 is one type of content creation system that generates 
television-like cartoons for business, entertainment or education. In these low 
bandwidth media systems, each of the graphical objects ("characters") of the 
presentation are well defined, and multimedia presentations are often developed wherein 

10 these characters move from segment to segment or from frame to frame. Accordingly, 
the rendering requirements of the presentation vary greatly, generally becoming quite 
high when a new character is introduced. Due to this variation, the amount of data that 
would ensure uninterrupted playback if received prior to rendering the streamable data 
object 107 from its beginning, may differ widely from the amount needed to ensure 

15 uninterrupted playback from a secondary position, such as the middle of the streamable 
data object 107. 

Therefore, there is a need for a system and a method that provides for the 
uninterrupted presentation of streamable data objects, including those streamable media 
objects imposing rendering requirements that vary over time. Moreover, the system and 

20 the method should not require the client computer 1 12 to store the entire streamable date 
object before starting the presentation. Further, the system and the method should 
provide for the uninterrupted rendering of a streamable data object 107 for each of the 
possible playback positions of the streamable data object 107. 

Summary of the Invention 

25 The system and method of the present invention have several features, no single 

one of which is solely responsible for its desirable attributes. Without limiting the 
scope of this invention as expressed by the claims that follow, its more prominent 
features will now be discussed briefly. After considering this discussion, and 
particulariy after reading the section entitled "Detailed Description of the Invention" one 

30 will understand how the features of this invention provide several advantages over 
traditional streaming systems. 
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The invention includes a streaming media system that provides for the 
calculation of a pre-roU for each segment of a multimedia data object. Using the 
calculated pre-roU, a client computer can render a multimedia presentation starting at 
each playable position of the multimedia data object with interruption. Further, the 
5 streaming media system of the invention minimizes the amount of buffering that is 
needed before rendering a presentation based upon the multimedia data object. 

Further, the streaming media system of the invention has a graphical utility that 
enables a user to re-arrange the presentation of the multimedia object such that the pre- 
roU of the multimedia data object is minimized. 

10 One embodiment of the present invention is a method of calculating a pre-roll 

for a plurality of segments in a muhimedia data object. The method comprises 
identifying a plurality of segments in the multimedia data object, calculating a pre-roll 
for each of the plurality of segments. An aspect of the embodiment further comprises 
storing the pre-roll for each of the plurality of segments in the multimedia data object. 

15 Another aspect of the embodiment is that wherein calculating a pre-roll includes 
determining a target bit rate of a client computer. A further, aspect of the embodiment is 
one wherein calculating a pre-roll includes determining the size of each of the plurality 
of segments, and wherein the pre-roll for each of the plurality of segments corresponds 
to a minimum amount of data to send to the client computer to enable the client 

20 computer to render the multimedia object without interruption starting from the 
respective segment. Still another aspect of the embodiment is one wherein the data of 
each segment is examined only once. 

Another embodiment of the invention a system for calculating a pre-roll for a 
plurality of segments in a multimedia data object. The system comprises means for 

25 identifying a plurality of segments in the multimedia data object and means for 
calculating a pre-roll for each of the plurality of segments, the pre-roll for each of the 
plurality of segments corresponds to a minimum amoimt of data to send to a client 
computer to enable the client computer to render the multimedia data object without 
interruption starting from the respective segment. In another aspect, the embodiment 

30 further comprises means for generating the multimedia data object. In still another 
aspect, the embodiment further comprises means for rendering the multimedia data 
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object into a multimedia presentation. In yet another aspect, the embodiment further 
comprises means for displaying the pre-roll. In one more aspect, the embodiment 
further comprises means for storing the pre-roll for each of the plurality of segments in 
the multimedia data object. 
5 Yet another embodiment of the invention is a system for streaming a muhimedia 

data object across a network. The embodiment comprises (1) a streaming media server 
operably connected to the network; (2) a client computer operably connected to the 
network; and (3) a multimedia data object including a segment index, the segment index 
defining a pre-roll, the pre-roll indicating a transmission of data from the streaming 

10 media server to the client computer that occurs before the client computer displays a 
presentation that is represented by the multimedia data object, the presentation starting 
at a selected segment of the muhimedia data object. Another aspect of the embodiment 
further comprises a pre-roll calculation program that creates the segment index. Still 
another aspect of the embodiment is one wherein the selected segment is a frame. 

1 5 Still another embodiment of the invention is a method of analyzing a multimedia 

data object that is adapted for rendering on a computer. The method comprises 
determining the rendering requirements of at least a portion of a multimedia data object; 
and displaying the rendering requirements of the multimedia data object. Another 
aspect of the embodiment is one wherein displaying the rendering requirements further 

20 includes plotting a consumption graph. Still another aspect of the embodiment is one 
wherein determining the rendering requirements of at least a portion of the multimedia 
data object includes identifying a plurality of segments within the multimedia data 
object; and determining the amount of data within each of the segments of the 
multimedia data object. 

25 One more embodiment of the present invention is a method of analyzing a 

multimedia data object that is adapted for rendering by a computer. The method 
comprises (1) determining a commxmication bit rate of a second computer; (2) 
determining the rendering requirements of the second computer for displaying at least a 
portion of the multimedia data object; (3) determining a pre-roll for said portion of the 

30 multimedia data object, the pre-roll corresponding to a minimum amount of data to send 
to the second computer to enable the second computer to render the multimedia object 
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without interruption starting from the portion; and (4) displaying the pre-roll. Another 
aspect of the embodiment is one wherein the multimedia data object has a plurality of 
segments, wherein the method for determining the rendering requirements further 
includes determining the rendering requirements for each segment of the muhimedia 
5 data object, and wherein determining the pre-roU further includes determining a pre-roU 
for each segment of the multimedia data object. Yet another aspect of the embodiment 
further includes storing the pre-roll. Still another aspect further includes re-organizing 
the multimedia data object. 

Still another embodiment of the present invention a program storage device 

10 storing instructions that when executed by a computer perform a method. The method 
comprises (1) identifying at least two positions within a multimedia data object; (2) 
determining a communication bit rate of a second computer; (3) determining the 
rendering requirements of the second computer for rendering the multimedia data object 
at each of the at least two positions; (4) determining a pre-roll for each of the at least 

15 two positions within the multimedia data object; and (5) storing the pre-roll for each of 
the at least two positions. 

Another embodiment of the present invention is a method of streaming a 
multimedia data object across a network. The method comprises (1) receiving a request 
for the transmission of a selected portion of a multimedia data object; (2) determining a 

20 pre-roll for the selected portion of the multimedia data object, the determining based in 
part on the rendering requirements of the multimedia data object and the transmission 
rate of the a client computer; (3) transmitting the multimedia data object to a client 
computer; and (4) signaling the client computer to start rendering the multimedia data 
object. In another aspect of the embodiment, the method further comprises rendering 

25 the multimedia data object. 

Still another embodiment of the present invention is a multimedia data object 
stored on a computer readable medium. The multimedia data object comprises a 
plurality of segments; and a segment index, the segment index indicating a pre-roll for 
each of the plurality of segments. One aspect of the embodiment is one wherein the 

30 segment index further comprises the position of each segment within the multimedia 
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data object. Another aspect of the embodiment is one wherein each of the segments is a 
frame. 

Another embodiment of the present invention is a streamable muhimedia object 
stored on a computer readable medium. The streamable multimedia object comprises a 
5 multimedia file having a sequence of segments; and a table having an entry for each of a 
plurality of said segments, each of said entries indicating a minimum amount of data of 
the multimedia file that must be sent to a client computer before the client computer 
begins rendering, to enable the client computer to render the multimedia file 
uninterrupted starting at a segment corresponding to the respective entry, the client 
1 0 computer receiving data at a given bit rate. 

One more embodiment of the present invention is a method to facilitate the 
uninterrupted play of a multimedia file having a sequence of segments by a client 
computer over a narrow bandwidth network. The method comprises the acts of 
identifying a plurality of segments within said multimedia file; and determining, for 
15 each of said plurality of segments, the minimum amount of data of the muhimedia file 
that must be sent to a client computer before the client computer begins rendering, to 
enable the client computer to render the multimedia file uninterrupted starting at a 
respective one of each of said segments, the client computer receiving data at a given bit 
rate. 

20 Still one more embodiment of the present invention is a multimedia streaming 

system. The system comprises (1) a network; (2) a multimedia data object; (3) a 
streaming media server operably connected to the network and the multimedia data 
object, the streaming media server adapted to stream the multimedia data object over the 
network; (4) a client computer operably connected to the network, the client computer 

25 adapted to receive the multimedia data object over a network, the client computer 

having a rendering program that is adapted to render the multimedia data object into a 
multimedia presentation; and (5) a pre-roU calculation program that generates a segment 
index, the segment index defining a pre-roU, the pre-roU indicating a data transmission 
that occurs before the client computer renders a multimedia presentation represented by 

30 the multimedia data object, the presentation starting at a selected segment of the 
multimedia data object. 
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Brief Description of the Drawings 
Figure 1 is a high-level block diagram of a streamable media system having a 
streamable media server and at least one client computer; 

Figure 2 is a flowchart of a conventional data streaming process; 
5 Figure 3 is a bit consumption graph illustrating the number of bits that are used 

by a traditional rendering program on the client computer of Figure 1 during the 
presentation of a streamable data object; 

Figure 4 is the bit consumption graph of Figure 3 fiirther illustrating the 
transmission rate between the streamable media server and the client computer of Figure 
10 1; 

Figure 5 is a high level functional block diagram illustrating the input and the 
output to a pre-roll calculation program in one embodiment of the invention; 

Figure 6 is a flowchart illustrating the pre-roll calculation process of the pre-roll 
calculation program of Figure 5; 
1 5 Figure 7 is a flowchart illustrating a streaming media process in one embodiment 

of the present invention for streaming a multimedia data object; 

Figure 8 is a flowchart illustrating a process in one embodiment of the present 
invention for calculating an appropriate pre-roll for each segment of a multimedia data 
object; 

20 Figure 9 is a screen display illustrating an exemplary user interface having a bit 

consumption graph; and 

Figure 10 is the screen display of Figure 9 fiirther illustrating an exemplary pull 
down menu that enables adjustment the target bit rate for the client computer. 

Detailed Description of the Invention 
25 The following detailed description is directed to certain specific embodiments of 

the invention. However, the invention can be embodied in a multitude of different ways 
as defined and covered by the claims. In this description, reference is made to the 
drawings wherein like parts are designated with like numerals throughout. 



-10- 



System Overview 

Figure 1 illustrates a high-level block diagram of a streaming media system that 
can be used with an embodiment of the present invention. The streaming media system 
includes an encoder server 106, a content creation station 108, a streaming media server 
5 110, and at least one client computer 112. 

It is noted that the encoding server 106, the content creation station 108, the 
streaming media server 1 10, and the client computer 1 12 may each be any conventional 
general purpose computer using one or more microprocessors, such as a Pentium 
processor, a Pentium II processor, a Pentium Pro processor, an xx86 processor, an 8051 

1 0 processor, a MIPS processor, a Power PC processor, or an ALPHA processor. 

In one embodiment of the present invention, the streaming media system 
includes a pre-roU calculation program 502 (Figure 5). The pre-roU calculation program 
502 determines an appropriate pre-roll transmission period for each segment of a 
multimedia data object 500 (Figure 5). The pre-roll corresponds to a length of time for 

15 receiving data before beginning the rendering of the data, or indicates an amount of 
data, or both, the pre-roll allowing for the uninterrupted rendering of the data from any 
selected beginning point to completion. 

A segment is defined as an identifiable portion of a multimedia object. A 
segment may be determined based upon the selected type of the multimedia object. For 

20 example, if the multimedia object uses frames as a presentation building block, as does 
a MacroMedia Shockwave Flash 2.0 muhimedia file, each segment may by defined as a 
fi-ame. Alternatively, if the multimedia object uses another measurement, such as time, 
for arranging the sequencing of the display of the multimedia data object, each segment 
may represent such a measurement. The purpose of identifying segments for the 

25 multimedia data object 500 is to be able to allow the user to render the multimedia data 

object 500 at a selected position within the multimedia data object 500. 

The pre-roll calculation program 502 (Figure 5) uses a multimedia data object 
500 (Figure 5) to create a modified muhimedia data object 504 that includes pre-roll 
information. It is noted that the multimedia data object 500 may represent any type of 

30 multimedia presentation, such as a MacroMedia Shockwave Flash 2.0 file or a 
RealNetworks RealPix presentation. 



-11- 



In one embodiment, the pre-roll calculation program 502 (Figure 5) includes 
various modules that are typically separately compiled and linked into a single executable 
program. Accordingly, the pre-roll calculation program 502 may be optionally integrated 
into one or more programs that reside and execute on any of the computing devices in the 
streaming media system, such as the encoder server 106 (Figure 1), the content creation 
station 108 (Figure 1), or the streaming media server 1 10 (Figure 1). 

The pre-roll calculation program 502 (Figure 5) may be written in any 
programming language such as C, C++, BASIC, Pascal, JAVA, and FORTRAN. C, 
C++, BASIC, Pascal, JAVA, and FORTRAN are industry standard programming 
languages for which, many commercial compilers can be used to create executable code. 
Further, the pre-roll calculation program 502 may be used in connection with various 
operating systems such as: UNIX, Solaris, Disk Operating System (DOS), OS/2, 
Windows 3.X, Windows 95, Windows 98, and Windows NT. 

In addition to the pre-roll calculation program 502, one embodiment of the 
present invention includes a server program (not shown) that resides on the streaming 
media server 110 and a client rendering program (not shown) that resides on the client 
computer 112. 

Method of Operation 

Figure 6 illustrates a high-level process of one embodiment of the present 
invention for creating and transmitting the multimedia data object 500 to a user. In a state 
600, a user creates the multimedia data object 500 (Figure 5) (e.g., a MacroMedia 
Shockwave Flash 2.0 file). The multimedia data object 500 may be created by the content 
creation station 108 or may be derived from a digitized signal from the microphone 102 
(Figure 1), the camera 104 (Figure 1), or other input device. 

Next, at a state 602, the pre-roll calculation program 502 (Figure 5) determines an 
appropriate pre-roll (or buffer or duration) for every segment of the multimedia data object 
500. The process for determining the pre-roll is described in fiuther detail in reference to 
Figures 7 and 8. However, in summation, the pre-roll calculation program 502 iterates 
through each segment of the multimedia data object 500 (Figure 5) and determines 
necessary pre-roll that is needed to play the entire multimedia data object 500 wdthout 
interruption starting at any selected segment. After the pre-roll information is determined, 
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the pre-roll calculation program 502 creates the multimedia data object 504 (Figure 5) that 
incorporates the pre-roll information. 

Continuing to a state 604, the client computer 1 12 requests the transmission of the 
multimedia object 500 starting at a user-selected segment. The transmission process is 
described in further detail below in a section entitled "Streaming Multimedia data 
Objects." In one embodiment, a user interface of a client rendering program presents a 
text box in which a user may enter via keystrokes a frame number of a frame-based 
multimedia object. In that embodiment, the client repdering program transmits a data 
packet including a request for the frame-based multimedia object and the frame number at 
which to begin playback. 

Moving to a state 606, the streaming media server 110 receives the request, 
accesses the multimedia data object at the specified segment, and begins transmitting data 
corresponding to the specified segment of the multimedia data object 504 (Figure 5) to the 
client computer 1 12 (Figure 1). It is noted that the multimedia data object 504 should be 
placed on storage media, such as a hard disk accessible by the streaming media server 1 10 
(Figure 1), so that the multimedia data object 504 can be transmitted to the client 
computer 1 12 (Figure 1). Further, it is also noted that, in one embodiment, as part of the 
process of sending the multimedia data object 504 to the client computer 112, the 
streaming media server 1 12 breaks the multimedia data object into packets and transmits 
("streams") the packets across the network 100 according to a flow control scheme. One 
of ordinary skill in the art vsdll appreciate that numerous flow control schemes exist, and 
the present invention is in no way limited by any flow control scheme. 

Still referring to Figure 6, at a state 608, the client computer 1 12 (Figure 1) waits 
for the full data transmission indicated by the pre-roll for the selected segment. By 
waiting for the pre-roll and receiving the associated data, the client computer 112 has 
enough data from the multimedia data object 504 (Figure 5) such that the client computer 
112 can begin playing the multimedia data object 504 without interruption starting at the 
user-selected segment. It is noted that the present invention may prevent the premature 
playback of the multimedia data object 504 by employing any one of a number of standard 
synchronization schemes. 
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Finally, at a state 610, the client computer 112 (Figure 1), starting at the user- 
selected segment, presents the multimedia presentation associated with the multimedia file 
to the user. While the presentation is in progress, the client computer 1 12 continues to 
receive the remainder of the multimedia data object 504 from the streamable media server 
5 110 (Figure 1). Advantageously, during the remainder of the presentation, the client 

computer 112 never interrupts the presentation to wait for additional data. The client 
computer 1 12 has pre-rolled enough of the multimedia data object 504 such that the client 
computer 1 12 always has in its possession at least enough data to continue the playback or 
rendering of the muhimedia presentation without interruption. 

10 Figure 7 is a flowchart illustrating the process for calculating the pre-roU of each 

segment of the multimedia data object 500 (Figure 5) in accordance with one 
embodiment of the present invention. Figure 7 illustrates some of the steps that occur in 
state 602 of Figure 6 in further detail. Exemplary screen shots of the pre-roll calculation 
program 1 02 are described in further detail in reference to Figures 9 and 10. 

1 5 Before beginning at a start state 700, the user of the pre-roll calculation program 

502 (Figure 5) has obtained or created a multimedia data object, such as the multimedia 
data object 500. Further, the user wants to make the multimedia data object available 
for streaming the across the network 100 (Figure 1). Using the pre-roll calculation 
program 502, the user can generate a consumption graph to visualize the expected 

20 presentation requirements of the multimedia data object over time in association with an 
amount of transmitted data. In other words, from the consumption graph, the user can 
determine pre-roll for any segment of the multimedia data object given a particular 
target bit rate at which data may be transmitted to a client computer 112. Further, the 
user can calculate the pre-roll at various target bit rates of the connection between the 

25 streaming media server 110 (Figure 1) and the client computer 1 12 (Figure 1). 

After viewing the consumption graph, the user can identify particular segments 
of the muhimedia data object 500 that consume more bandv^dth than the other 
segments. Using the information from the consumption graph, the user can reorganize 
the layout of the multimedia data object 500 so as to reduce the bandwidth requirements 

30 of the identified segments. The user can again use the pre-roll calculation program 502 
to determine if the reorganized multimedia data object 500 has a more evenly 
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distributed consumption requirement for each of the segments, and can also determine if 
the pre-roll for the muhimedia data object has increased or decreased, either generally or 
on a segment-by-segment basis. 

Starting at the state 700 of Figure 7, the user executes the pre-roll calculation 
5 program 502 (Figure 5). Next, at a state 702, the pre-roll calculation program 502 
requests the name of the streamable data object 500 (Figure 5). Continuing to a state 
704, the pre-roll calculation program 502 parses the selected muhimedia data object 500 
into the memory of the computer executing the pre-roll calculation program 502. 

Moving to a state 706, the pre-roll calculation program 502 (Figure 5) creates a 

10 segment information table based upon the multimedia data object 500 (Figure 5). The 
segment information table includes information about each segment in the multimedia 
data object. For each segment in the multimedia data object 500, the pre-roll calculation 
program 502 generates a segment data structure that is stored in the segment 
information table. Each data structure in the segment information table contains the 

15 position and length or size of each segment within the multimedia data object 500. The 
segment information table may be structured as an array of segment data structures, or a 
linked list of segment data structures, or some other type of data structure. 

Next, at a state 708, the pre-roll calculation program 502 (Figure 5) calculates 
the number of bits that are consumed by the client computer 112 (Figure 1) for each 

20 segment of the multimedia data object 500. Typically this information is readily 
calculable based upon the information in the multimedia data object 500. For example, 
in a MacroMedia Shockwave Flash 2.0 file, the number of bits that are consumed by the 
client computer 112 is the number of bits that are contained in each frame of the 
MacroMedia Shockwave Flash 2.0 file. Proceeding to a state 710, the pre-roll 

25 calculation program 502 displays a plot of the consumption of the multimedia data 
object 500 by the client computer 1 12 over time. 

At a next state 712, the pre-roll calculation program 502 provides an option 
menu screen, such as is shovm in Figure 9, to the user. Further, the pre-roll calculation 
program 502 displays various options and other items of information related to the 

30 multimedia data object 500. The three primary fimctions offered by the option menu 
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include (i) adjusting the target bit rate, (ii) re-allocating the bandwidth for the 
streamable data, and (iii) exiting the program. 

If the user requests to adjust target bit rate of the client computer 1 12 (Figure 1), 
the pre-roll calculation program 502 (Figure 5) proceeds to a state 714. The pre-roll 
calculation program 502 uses the target bit rate to calculate the total amount of 
bandwidth available for transmission of information from the streaming media server 
110 (Figure 1) to the client computer 1 12. The multimedia data object 504 (Figure 5) 
may not be the only data object that is to be distributed to the user. 

For example, assume that the multimedia data object 500 is a MacroMedia 
Shockwave Flash 2.0 animation file that is designed to be accompanied by some audio. 
Further assume that client computer 1 12 has a target bit rate of 28.8 Kbps. The provider 
of the MacroMedia Shockwave Flash 2.0 animation file should determine how much 
pre-roll is needed for a specified bit rate yet allowing some further bandwidth for the 
transmission of the accompanying audio file. If the user allocates 12 Kbps of the 
bandwidth for the MacroMedia Shockwave Flash 2.0 file, around 8 Kbps will be 
available for the user to transmit the accompanying audio file (assuming, for example, 
that another 8 Kbps is consumed by transmission overhead). 

In another state 715, the pre-roll calculation program 502 calculates the amount 
of available bandwidth for other applications after the adjustment of target bit rate. The 
available bandwidth is defined to be equal to the adjusted bandwidth minus the selected 
bit rate. Using the example from above, assuming the target bit rate was adjusted fi-om 
28.8 Kbps to 56 Kbps (34 Kbps after deducting 22 Kbps for transmission overhead), the 
pre-roll calculation program 502 would subtract the selected bit rate 12 Kbps fi-om the 
adjusted bit rate (including transmission overhead) of 34 Kbps to derive an available bit 
rate of 22 Kbps. The pre-roll calculation program 502 then returns to the state 708 to 
recalculate the bit consumption by the client computer 112. 

Referring again to the state 712, if the user chose to re-allocated the bandwidth, 
then, in a state 716, the user adjusts the distribution of bandwidth between the 
multimedia data object 500 and other transmissions. The more bandwidth that is 
allocated to the multimedia data object 500, the shorter is the required pre-roll. 



-16- 



Conversely, a decrease in the amount of bandwidth allocated to the multimedia data 
object 500 increases the pre-roU. 

Referring yet again to the state 712, if the user chooses to exit, the pre-roU 
calculation program 502 (Figure 5) exits the options menu in a state 718. Next, at a 
5 state 720, the pre-roll calculation program 502 calculates the pre-roll for each segment 
in the segment information table. The process for calculating the pre-roll for each frame 
is described below in further detail in reference to Figure 8. However, in summary, the 
pre-roll calculation program 502 iterates (in reverse order) through each segment data 
structure in the segment information table, reads the segment information, and generates 

1 0 a segment index entry which is stored in a segment index. 

Each segment index entry includes two items of information related to the 
segment. First, the segment index entry includes an element that describes the position 
of the segment within the multimedia data object. Second, each segment index entry 
includes an element representing the length or amount of the pre-roll for rendering the 

1 5 data object starting at the selected segment. 

Finally, in the state 722, the pre-roll calculation program 502 (Figure 5) stores 
multimedia data object 500 (Figure 5) with a segment index, thereby creating the 
multimedia data object 504 (Figure 5). In one embodiment of the invention, the 
segment index is appended to the end of the multimedia data object 500. Thus, each 

20 segment index entry may be stored in consecutive order following the last byte in the 

multimedia data object 500. In one embodiment of the invention, each segment index 
entry is stored as a pair of integers, one integer for the pre-roll, the other integer for the 
position of the segment within the multimedia data object 500. 

It is noted that, in accordance v^th another embodiment of the present invention, 

25 the pre-roll calculation program 502 may also be adapted to calculate and store the 

required pre-roll for multiple target bit rates. Advantageously, this allows each client 
machine 1 12 to retrieve the multimedia data object 504 with a pre-roll most suited to its 
available bandwidth. 

Figure 8 illustrates an exemplary process for calculating the pre-roll for each 

30 segment of the multimedia data object 500 (Figure 5) in accordance with one 
embodiment of the present invention. Figure 8 illustrates the acts that occur in the state 
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720 of Figure 7 in further detail. The purpose of the pre-roll calculation is to determine 
for each segment in the multimedia data object 500 a pre-roll, the pre-roll indicating a 
quantum of data to send to the client computer 1 12 (Figure 1) before the client computer 
1 12 begins presenting the muhimedia data object 500 to the user. 

Starting at a state 800 in accordance with one embodiment of the present 
invention, the pre-roll calculation program 502 (Figure 5) iterates through each segment 
data structure in the segment information table (created in state 706 of Figure 7) starting 
at the last segment data structure in the segment information table. The pre-roll 
calculation program 502 executes states 802-810 for each segment data structure in the 
segment information table. 

Moving to a state 802, the pre-roll calculation program 502 accesses the first 
segment data structure from the segment information table (starting at the rear of the 
segment table). Advantageously, beginning at the rear of the segment table and 
processing the segments in reverse order in accordance with the disclosed embodiment 
enables pre-roll to be calculated for each segment during only a single pass through the 
information of the muhimedia object. Next, at a state 804, the pre-roll calculation 
program 502 gets the size of the segment from the segment data structure. 

Continuing to a state 806, the pre-roll calculation program 502, using Equation 
1, determines the amount of time that the streaming media server 104 sends data for a 
specified segment. 

RunningPreroU = 0; (first iteration). 

SPS = Segments per second. 

(1) RunningPreroU = RunningPreroU - 1000/SPS. 

The variable SPS is defined by the number of segments that the client computer 
112 needs per second for the display of the multimedia object. For example, if the 
client computer 112 needs 10 segments per second, RunningPreroU is decremented by 
100 milliseconds. 

Next, at a state 808, the pre-roU calculation program 502, using Equation 2, 
determines the pre-roll for the selected segment. 

Segmentsize = Length of a selected segment. 

TargetbitRate = Targeted bit rate of the client computer 1 12. 
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(2) RunningPreroll = RunningPreroll + (1000 * Segmentsize * 8 / 
TargetbitRate) 

It is noted that there are 8 bits in a byte. To convert the segment size, typically 
stored as bytes, into bits, the segment size is multiplied by 8. It is also noted that the 
target bit rate is stored in units of Kilobits per seconds. Thus, the pre-roll calculation 
program 502 determines the pre-roll for the selected segment. 

Moving to a state 810, if the value of RunningPreroll is negative (indicating that 
no pre-roll is necessary), the pre-roll calculation program 502 (Figure 5) records in the 
segment index that the selected segment needs zero pre-roll. If the value of 
RunningPreroll is zero or greater, the pre-roll calculation program 502 stores the 
RunningPreroll plus a deha amount for a transmission overhead. In one embodiment of 
the invention, the delta amount is 750 milliseconds. The transmission overhead is the 
duration of time it takes to account for transmitting the multimedia object 500 across the 
network 100. The pre-roll is stored in units of milliseconds. In another embodiment, 
the pre-roll is stored as a number of bits or other units corresponding to an amount of 
data. The process flow then returns to the state 802 to get the next segment data 
structure from the segment information table. The process continues through states 802- 
810 for each segment until each segment has an associated pre-roll value. 

Table 2 illustrates the state of the variables of Equations 1 and 2 after they have 
been executed for three exemplary segments (note, that, for the segments one, two and 
three of Table 2, segment two occurs prior to segment one, and segment three occurs 
prior to segment two during playback of the multimedia object). It is noted that the 
actual value of the variables will vary from implementation to implementation. 
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Table 2 



Variables 


Segment 
One 

(Equation 


Segment 
One 

(Equation 
2) 


Segment 
Two 

(Equation 


Segment 
Two 

(Equation 
2) 


Segment 

Three 

(Equation 

1) 


Segment 
three 
(Equation 
2) 


Running 

Preroll 

(ms) 


-100 


400 


300 


340 


240 


280 


Framesize 
(bytes) 




1000 




100 




100 


TargetBit 
Rate (bps) 




20,000 




20,000 




20,000 


Segments 
Per 

Second 


10 




10 




10 





Figure 9 is a screen display of an exemplary embodiment of the pre-roU 
calculation program 502 (Figure 5). The option menu screen 1012 includes numerous 
5 pull-down menus, data fields, and slide bars that are commonly used in many window- 
based applications. It is noted that the positioning and size of the pull-down menu 
screens, the data fields, and the slide bars may be varied in alternative embodiments of 
the invention. 

The option menu screen 1012 includes consumption graph 1010. A vertical axis 
10 of the graph 1010 indicates the number of bits that are consumed by the client computer 
112 for a selected time period. The option menu screen 1012 also has a bit rate input 
window 1014 and a calculated buffer time (pre-roU) display window 1016. The bit rate 
display window 1014 indicates the bandwidth that the provider of the multimedia data 
object wants to dedicate to the multimedia data object 500 (Figure 5). 
15 In one embodiment of the present invention, the buffer time display window 

1016 displays a pre-roU buffer time that permits the presentation associated with the 
multimedia data object to be played without interruptions. It is noted that the pre-roll 
buffer time is also graphically illustrated as a part of the consumption graph 1010. The 
pre-roll buffer time is illustrated by a line 1018, the location of the line 1018 on the 
20 horizontal axis (representing time) indicating the amount of pre-roll. 

Figure 10 is an exemplary screen display 1 100 that further illustrates a pull down 
sub menu 1 102. Using the pull down sub menu 1 102, the user can adjust the target bit 
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rate for the client computer 1 12 (Figure 1). By adjusting the target bit rate, the pre-roll 
calculation for the multimedia data object 500 is adjusted as well as the available 
bandwidth for other applications. The effect of adjusting the bit rate is explained above 
in further detail in reference to state 714 of Figure 7. 

Streaming Multimedia Data Objects 

The process for streaming the multimedia data object 504 (Figure 5) having pre- 
roll information is disclosed below. First, the multimedia data object 504 is stored in a 
computer readable medium accessible by the streaming media server 100 (Figure 1) so 
that the streaming media server 110 can transmit the multimedia data object 504 to the 
client computer 112. Alternatively, the multimedia data object 504 could be stored on 
media readable by any computer that is accessible by the streaming media server 1 10. 

Next, the client computer 1 12 (Figure 1) signals to the streaming media server 1 10 
that the client computer 112 wants to receive and render a selected segment of the 
multimedia data object 504 (Figure 5). The streaming media server 110 then retrieves the 
header information for the multimedia data object 504. The header information can vary 
for each type of multimedia data object. Typical header information includes the title of 
the multimedia data object 504, the author of the multimedia data object 504, and the type 
of the multimedia data object 504. However, in one embodiment of the invention, the 
header also includes the pre-roll for the whole multimedia-media data object 504. 

The client computer 1 12 then transmits to the streaming media computer 1 10 the 
number of the segment in the multimedia data object 110 that the client computer 1 12 has 
requested for rendering. In one embodiment of the invention, if the requested segment is 
zero, the client computer 112 waits the amount of time indicated in the header before 
rendering the multimedia data object 504. However, if the requested segment was greater 
than zero, the streaming media server 110 searches the segment index in the muhimedia 
data object 504 to find the pre-roll for the selected segment. It is noted that depending on 
the embodiment of the invention, the pre-roll information is stored before, after, or in the 
middle, or intermixed with the multimedia data object 500. 

Thus, in an embodiment wherein the pre-roll information is at the end of the 
multimedia data object 504, the streaming media server 1 10 uses the header information 
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of the multimedia data object 504 to determine the last byte of the multimedia data object 
504. The streaming media server then indexes into the segment index (which is 
positioned following the last byte of the multimedia data object) to determine the pre-roll 
of the segment that was selected by the client computer 1 12. 

The streammg media server 1 10 then transmits the multimedia data object 504 in 
packets having a time stamp of zero. The client computer 1 12 receives the packets, but 
performs no rendering of the received data in the packets until the time stamp is greater 
than zero. Once the streaming media server 1 10 determines that the time indicated by the 
pre-roll for the selected segment has passed, the streaming media server 110 starts 
incrementing the time stamps. 

After the client computer 112 receives a packet with a non-zero time stamp, the 
client computer 1 12 starts rendering the received data from the multimedia data object 504 
into a presentation. While the presentation is being rendered, the client computer 112 
continues to receive the remainder of the multimedia data object 500 that has not yet been 
transmitted. Having received data without rendering it for the pre-roll duration, the client 
computer 1 12 displays the presentation v^thout ever pausing due to insuflRcient data. 

Advantageously, the streaming media system of the invention provides for the 
determination of a pre-roll for each segment of a multimedia data object 504. The pre- 
calculation of the pre-roll insures that a client computer, such as the client computer 1 12 
(Figure 1), has enough data from the multimedia object 504 such that playback or 
rendering of the muhimedia data object 504 is uninterrupted. Further, the user can seek 
or jump to any portion (segment) of the presentation represented by the multimedia data 
object 500 and render the multimedia data object 500 without subsequent interruptions. 

In another embodiment, the present invention advantageously adjusts pre-roll to 
variations in the actual bit rate or bandwidth available to the client computer 112, 
adjusting the pre-roll down when bandwidth increases, and adjusting the pre-roll up 
when bandwidth decreases. Thus, the adjusted pre-roll varies inversely with changes in 
bandwidth. Accordingly, the streaming media server 110 first determines or estimates 
the actual bit rate at which the client computer 1 12 receives data. It will be appreciated 
by one of ordinary skill in the art that numerous techniques exist for estimating 
bandwidth. After estimating the available bandwidth to be x Kbps, for example, the 
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streaming media server 1 10 calculates the ratio (y Kbps)/(x Kbps) where y Kbps is the 
bit rate at which pre-roll was calculated by the pre-roll calculation program. The 
streaming media server 110 multiplies the pre-roll for a selected segment by the 
calculated ratio to arrive at an adjusted pre-roll. The streaming media server 1 10 then 
begins to transmit packets of the multimedia data object 504 to the client computer, 
placing time stamp values of zero on the packets until the time indicated by the adjusted 
pre-roll has passed, at which time the streaming media server 110 begins incrementing 
the values of the time stamps. It will be appreciated that before calculating the ratio, the 
streaming media server 110 may subtract out proportional portions of the bandwidth 
allocated to transmission overhead or to the transmission of other data such as, for 
example, other media objects. The disclosed ratio technique represents only one 
manner in which the present invention may calculate an adjusted pre-roll based on 
variations in bandwidth available to a client computer 1 12. 

Further, the present invention provides a graphical utility providing pre-roll 
information about a multimedia data object 504 such that a user of the utility can easily 
perceive any bandwidth intensive portions of a multimedia data object. The user can 
then re-arrange or modify the multimedia data object to minimize delays associated with 
random accesses to the multimedia data object. 

While the above detailed description has shown, described, and pointed out 
novel features of the invention as applied to various embodiments, it will be understood 
that various omissions, substitutions, and changes in the form and details of the device 
or process illustrated may be made by those skilled in the art without departing from the 
spirit of the invention. The scope of the invention is indicated by the appended claims 
rather than by the foregoing description. All changes which come within the meaning 
and range of equivalency of the claims are to be embraced within their scope. 
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